Skip to main content

Machine Learning

Machine learning is a subset of artificial intelligence, it is the study of computer algorithms that improve automatically through experience. Machine learning algorithms build a mathematical model based on sample data, known as "training data", in order to make predictions or decisions without being explicitly programmed to perform the task.

The reason why we do machine learning is that we want to have a model that predict/classify/... but notice, every actual fact may also exists errors so that we may not have a perfect model to predict the outcome.

In machine learning, we assume there exist a perfect function ff that we can use the given XX to get value exactly same as YY. Formally, let XX be the input event, YY be the output outcome, such function ff represent some magic like Y=f(X)+ϵY = f(X) + \epsilon, ϵ\epsilon represent the error terms.

Denote f^\hat f be a estimate for ff (may not close), the predict outcome Y^\hat Y given by the f^(X)\hat f(X). We can found that how close the predict outcome Y^\hat Y to the real YY by the expectation formula.

E(Y−Y^)2=E[f(X)+ϵ−f^(X)]=[f(X)−f^(X)]2+Var(ϵ)E(Y - \hat Y)^2 = E[f(X) + \epsilon - \hat f(X)] = [f(X) - \hat f(X)]^2 + Var(\epsilon)

  • Var(ϵ)Var(\epsilon) is the nature part of the error which we can't avoid and it is irreducible
  • [f(X)−f^(X)]2[f(X) - \hat f(X)]^2 is the part of the error that we can avoid by choosing a better model f^\hat f

The goal of machine learning is to minimize the [f(X)−f^(X)]2[f(X) - \hat f(X)]^2 part of the error to get a better model f^\hat f, to generalize the model, we have $

Types of Machine Learning​

The main types of machine learning are:

  • Supervised Learning: learning from labeled data
  • Unsupervised Learning: learning from unlabeled data
    • find groups of samples/features that behave similarly or find patterns in the data by itself
    • difficulty to assess model and to quantify how well this learning doing.
    • Useful as pre-processing step for supervised learning
  • Reinforcement Learning: learning from feedback

We also describe machine learning model into two categories: parametric and non-parametric.

  • Parametric: make assumption about the function ff and then fit the model to the data
    • the choose(guess) model may not always match the true unknown form of ff. If we want to get closer model, we may need more flexible model, but may lead to overfitting, follow the errors, noise.
  • Non-parametric: avoiding the assumption of function ff, it provide a wider range of possible shapes for ff.
    • need far more observations/(training data) than the parametric approach to obtain an accurate estimate for ff.